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This paper introduces a version of empirical likelihood based on 
the periodogram and spectral estimating equations. This formula- 
tion handles dependent data through a data transformation (i.e., a 
Fourier transform) and is developed in terms of the spectral distribu- 
tion rather than a time domain probability distribution. The asymp- 
totic properties of frequency domain empirical likelihood are studied 
for linear time processes exhibiting both short- and long-range de- 
pendence. The method results in likelihood ratios which can be used 
to build nonparametric, asymptotically correct confidence regions for 
a class of normalized (or ratio) spectral parameters, including auto- 
correlations. Maximum empirical likelihood estimators are possible, 
as well as tests of spectral moment conditions. The methodology can 
be applied to several inference problems such as Whittle estimation 
and goodness-of-fit testing. 

1. Introduction. The main contribution of this paper is a new formu- 
lation of empirical likelihood (EL) for inference with two fundamentally 
different types of dependent data: time series exhibiting either short-range 
dependence (SRD) or long-range dependence (LRD). Let {Xt},t G Z, be a 
stationary sequence of random variables with mean /x and spectral density 
/ on n = [— vr, vr] , where 

(1) /(A)~C7(a)|A|-", A^O, 

for a G [0, 1) and a constant C{a) > involving a (with ~ indicating that the 
terms have a ratio of one in the limit). When a = 0, we classify the process 
{Xt} as short-range dependent (SRD). For a > 0, the process will be called 
long-range dependent (LRD). This classification resembles that of [25] and 
encompasses the formulation of LRD described in [4, 40]. 
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Originally proposed by [33, 34] for independent samples, EL allows for 
nonparametric likelihood-based inference with a broad range of applications 
[36]. An important benefit of EL inference is that confidence regions for pa- 
rameters may be calibrated through log-likelihood ratios, without requiring 
any direct estimates of variance or skewness [19]. However, a difficulty with 
extending EL methods to dependent data is then to ensure that "correct" 
variance estimation occurs automatically within EL ratios under the data 
dependence structure. This is an important reason why the EL version for 
i.i.d. data from [34] does not apply to dependent data (see [23], page 2085). 

Recent extensions of EL to time series in [23, 31] have relied exclusively 
on a SRD structure with rapidly decreasing process correlations. In partic- 
ular, [23] provided a breakthrough formulation of EL for weakly dependent 
data based on data blocks rather than individual observations. Under SRD, 
the resulting blockwise EL ratios correctly perform variance estimation of 
sample means within their mechanics. Data blocking has also proven to be 
crucial in extending other nonparametric likelihoods to weakly dependent 
processes, such as the block bootstrap and resampling methods described in 
[26], Chapter 2. 

In comparison to weak dependence, the rate of decay of the covariance 
function r{k) = Cov{X j,Xj^k) is characteristically much slower under strong 
dependence a > 0, namely, 

(2) r{k) ~ C(a)A;-(i-") , k^oo, 

with a constant C{a) > 0, which is an alternative representation of LRD 
(with equivalence to (1) if the covariances converge quasimonotonically to 
zero; see page 1632 of [40]). This auto covariance behavior implies that sta- 
tistical procedures developed for SRD may not be applicable under LRD, 
often due to complications with variance estimation. For example, the mov- 
ing block bootstrap is known to be invalid under strong dependence for 
inference on the process mean FiXt = /x [24] , partly because (2) implies that 
the variance Var(X„) = 0(n~^+") of a size n sample mean exhibits a slower, 
unknown rate of decay compared to the 0(?i~^) rate associated with SRD 
data. For this reason, the blockwise EL formulation of [23] will also break 
down under strong dependence for inference on the mean. 

In this paper, we formulate an EL based on the periodogram combined 
with certain estimating equations. Using this data transformation to weaken 
the underlying dependence structure, the resulting frequency domain empir- 
ical likelihood (FDEL) provides a common tool for nonparametric inference 
on both SRD and LRD time series. Because this EL version involves the 
spectral distribution of a time process rather than a time domain proba- 
bility distribution, inference is restricted to a class of normalized spectral 
parameters described in Section 2. The frequency domain bootstrap (FDB) 
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of [11], developed for SRD, targets the same class of parameters. Hence, for 
parameters not defined in terms of the spectral density (e.g., the process 
mean fi), the FDEL is inapplicable, while the time domain blockwise EL in 
[23] may still be valid if the process exhibits SRD (i.e., is valid for a larger 
class of parameters under weak dependence). 

Our main result is the asymptotic distribution of FDEL ratio statistics, 
which are shown to have limiting chi-square distributions under both SRD 
and LRD for setting confidence regions. That is, FDEL shares the strength 
of EL methods to incorporate "correct" variance estimation for spectral 
parameter inference automatically in its mechanics. For normalized spec- 
tral parameters where both the FDB and blockwise EL may be applicable 
under SRD (e.g., autocorrelations), the FDEL requires no kernel density es- 
timates of / (as with the FDB) and no block selection (as with the blockwise 
EL). Our FDEL results also refine some EL theory given in [31], wherein 
periodogram-based EL confidence regions for Whittle-type estimation with 
SRD linear processes were proposed. We additionally consider FDEL tests 
based on maximum EL estimation, which are helpful for assessing both pa- 
rameter conjectures and the validity of (spectral) moment conditions, as in 
the independent data EL formulation [34, 38]. 

The methodology presented here is applicable to linear processes with 
spectral densities satisfying (1), which includes two common models for LRD 
processes: the fractional Gaussian processes of [29] with spectral density 



(3) 



fnA^) = ,,:.2H+2 ^ os(^g-^/2)sin^(A/2) 



4o-2r(2i?- 1) 

oo 

^ |A/(27r) + fcri-2^, A en. 



1/2 < < 1, and the fractional autoregressive integrated moving average 
(FARIMA) processes of [1, 18, 21] with spectral density 



,2 



(4) Ap,,,.2(A) = 1^11-6^^1-2'^ 



A en, 



based on parameters < d < 1/2, p = (pi, . . . , pp), Q = [qi, ■ ■ ■ ^ Qq) with pQ = 
go = 1 and I = yj—l. These models fulfill (1) with a = 2H — 1 and a = 2d, 
respectively. 

The rest of the paper is organized as follows. Section 2 describes the role 
of spectral estimating equations for FDEL inference and provides several 
examples. In Section 3, we explain the construction of EL in the frequency 
domain. Section 4 contains the assumptions and the main results on the 
distribution of FDEL log-ratios for confidence region estimation and simple 
hypothesis testing. In Section 5, we consider maximum EL estimation in 
the frequency domain. We describe the application of FDEL to Whittle 
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estimation in Section 6, while Section 7 considers goodness-of-fit testing with 
FDEL. Section 8 offers some conclusions. Proofs of the results are given in 
Section 9 and the Appendix. 

2. Spectral estimating equations. Consider inference on a parameter 9 G 
C based on a time stretch Xi, . . . , X„. Following the EL framework of 
[38, 39] with i.i.d. data, we suppose that information about 6 exists through 
a system of general estimating equations. However, we will use the process 
spectral distribution to define moment conditions as follows. Let 



denote a vector of even, estimating functions with r >p. For the case r > p, 
the above functions are said to be "overidentifying" for 9. We assume that 
Gg satisfies the spectral moment condition 



for some known at the true value Oq of the parameter. As distribu- 

tional results in Section 4 indicate, we will typically require = 0, which 
places some restrictions on the types of spectral parameters considered. How- 
ever, the FDEL framework is valid for estimating normalized spectral means: 
6 = Gfd\/ Jq f dX, based on a vector function G. The FDB targets the 
same parameters under SRD; [11] comments on the importance, and often 
complete adequacy, of population information expressed in this ratio form. 
The FDEL construction in Section 3 combines the periodogram with the 
estimating equations in (6). 

2.1. Examples. We provide a few examples of useful estimating functions 
for inference, some of which satisfy (6) with = 0. 

Example 1 (Autocorrelations). Consider interest in the autocorrelation 
function p(-) at arbitrary lags mi, . . . ,mp, that is, 6 = (p(mi), . . . ,p{mp))', 
where 



One can select Gq{\) = (cos(miA), . . . , cos(mpA))' — 9 for autocorrelation 
inference, fulfilling (6) with ^A=Q and r =p. 

Example 2 (Spectral distribution function). For co £ [0,7r], denote the 
spectral distribution function by F{uj) = Jq /(A) dX. Suppose 9 = {F {ti) / F {tt) , 
. . . ,F(Tp) /F{tt) y for some ri , . . . G (0, vr) . This normalized parameter 9 of- 
ten sufficiently characterizes the spectral distribution F for testing purposes 



(5) 



^^(A) = (<7i,e(A), . . . ,9r,e(A))' : H x 6 ^ 



(6) 
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[9]. For inference on 6, we can pick Gg{X) = (1{A < ri}, . . . , 1{A < Tp})' — 9 
where !{•} denotes the indicator function. Then (6) holds with spectral 
mean M=0£MP. 

Example 3 (Goodness-of-fit tests). There has been increasing interest 
in frequency-domain-based tests to assess model adequacy [2, 37]. Consider a 
test involving a simple null hypothesis Hq '■ f = fo against an alternative Hi : 
/ / /o for some candidate density /q. With EL techniques, one immediate 
test for Hq is based on the function Go (A) = l//o(A) with spectral mean vr 
under Hq; here, we treat r = 1 and the dimension p of as 0. We show in 
Section 7 that this results in an EL ratio test which resembles a spectral 
goodness-of-fit test statistic proposed by [30] and shown by [3] to be useful 
for LRD Gaussian series. The more interesting and complicated problem of 
testing the hypothesis that / belongs to a given model family can also be 
addressed with FDEL, as discussed in Section 7. 

Example 4 (Whittle estimation). We denote a parametric collection of 
spectral densities by 

(7) J-={/,(A):^Ge} 

and assume the densities are positive on H and identifiable [e.g., for 6^9^ 
0, the Lebesgue measure of {A : fe{X) / /g(A)} is positive]. For fitting the 
model fg to the data. Whittle estimation [42] seeks the 0-value at which the 
theoretical "distance" measure 

(8) W{9) = (47r)-i /''{log^(^) + 

achieves its minimum [13]. The model class may be misspecified (possibly 
f ^ J-), but Whittle estimation aims for the density in T "closest" to /, as 
measured by W{9). 

To consider a particular parameterization of (7), suppose 

/q^ /0(A) = (t2/c^(A), 

9 = (cj2, i9')', C (0, 00) X RP-i, = (??i, . . . , §p_i)', 

with kernel density fc^, and that Kolmogorov's formula holds with (27r)~^ x 
/^j^ log/e)(A) dA = log[(T^/(27r)] (e.g., taking cr^ as the innovation variance 
in a linear model). The model class in (9) is commonly considered in the 
context of Whittle estimation for both SRD and LRD time series, including 
those LRD processes formulated in (3) and (4) (see [10, 15, 17, 20]). Under 
appropriate conditions, the true minimum argument 9q = {a^^'d'o)' of W{6) 
is determined by the stationary solution of dW {9) / 89 = or 

(10) r f{x){df^Hx)/d^}dx = o, r f(x)f^\x)dx = i, 

Jo Jo 
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where /^^(A) = 1/ fg{X). The moment conditions in (10) give a set of es- 
timating functions for FDEL inference on 9 defining the densities in (9). 
Namely, the choice 

fulfills (6). The FDB uses similar estimating equations for Whittle parameter 
inference [11]. To treat cr^ as a nuisance parameter, which is common for 
densities as in (9), estimating functions 

(12) GT{X) = dk^\X)/d^, M^, = OeW-\ 

provide structure for inference on the remaining parameters determining 
h in fe- 

3. Definition of frequency domain empirical likelihood. Denote the peri- 
odogram of the sequence Xi, . . . ,X„ by /n(A) = (27rn)^-'^| J2t=i 6xp(— ztA)p, 
A G n = [— 7r,7r], where ^ = \/— T. Using estimating functions Gg as in (5), 
the profile FDEL function for ^ G is given by 

f N N N ^ 

(13) Lni6)=suplY[wj:wj>0, ^Wj=Tr, ^WjGg{Xj)In{Xj) = m\ , 

[j=i j=i j=i J 

where Xj = 271 j/n, j G Z, are Fourier frequencies and N = [(n — l)/2j. 
Point masses Wj assigned to each ordinate Xj create a discrete measure on 
[0,7r] with the restriction that the integral of Ggin, based on this measure, 
equals Ai. The largest possible product of these point masses determines the 
FDEL function for 6 £Q. When the conditioning set in (13) is empty, we de- 
fine Ln{0) = — oo. If A4 is interior to the convex hull of {TrG0{Xj)In{Xj)}jLi, 
then Ln{0) is a positive constrained maximum solved by optimizing 



j=l \ j = l ) 

- Ni'{^pG6(Xj)ln{X,) - M\^, 



with Lagrange multipliers 7 and t = (ti,...,ir)' as in [33, 34]. Then (13) 
may be written as 



N 



p,ie)=N-^[l + t's{7rGeiX,)In{Xj)-M}r' G (0,1) 
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where tg is the stationary point of the function q{t) = X]j=i log(l + t'{7rGe(Aj) x 
Ini^j) — -^})- (See [34, 38] for further computational details on EL.) With- 
out the integral-type linear constraint in (13), 11^=1 has a maximum when 
each Wj = n/N, so we can form a profile EL ratio 

AT 

(15) Rnie) = Lnie)/{7TN-X = H + t'e{7TGgiX,)In{Xj) - M}]-\ 

i=i 

3.1. A density-based formulation of empirical likelihood. To help relate 
the EL results here to those in [31], we give an alternative, model-based 
formulation of FDEL. This version requires a density class J- as in (7) and 
involves approximating the expected value E(In(Xj)) with fg{Xj), using a 
density fg £ T . Namely, let 

( N N 

sup< Wj : Wj > 0, ^ Wj = TT, 
[j=i j=i 

Y,W,Gg{X,)[In{X,)-fgiXj)]=0\ 

We consider the densities fg and prospective functions Gg as dependent on 
the same parameters, which causes no loss of generality. An exact form for 
Ln.j^{0) can be deduced as with Ln{0), obtained by replacing In{Xj),M. with 
IniXj)-fg,0 in (14). 

Section 6 discusses the model-based EL ratio in (16) for refining results in 
[31] on confidence interval estimation of Whittle parameters. Additionally, 
this version of FDEL may be suitable for conducting goodness-of-fit tests 
with respect to a family of spectral densities. 

4. Main result: distribution of empiricallikelihood ratio. Before describ- 
ing the distributional properties of FDEL, we provide some assumptions on 
the time process under consideration and the potential vector of estimating 
functions Gg in (5). 

4.1. Assumptions. In the following, let Oq denote the unique (true) pa- 
rameter value which satisfies (6). 

Assumption A.l. {Xt} is a real- valued, linear process with a moving 
average representation of the form 

oo 

Xt = ii+ ^ bj£t-j, tez, 

j=~oo 



(16) 



RnAO) 
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where {et} are i.i.d. random variables with E(ej) = 0, E(e^) = cr^ > 0, E(ej ) < oo 
and fourth order cumulant denoted by K4^s = E(e^) — Su^ , {bt} is a sequence 
of constants satisfying Yjiei^l < °° ^^"^ ^0 = 1) and /(A) = (T^|6(A)p/(27r), 
A G n, with 6(A) = Z^jGZ ^j^*"''^- It is assumed that /(A) is continuous on 
(0,7r] and that /(A) < C|A|-°, A G H, for some a £ [0, 1), C > 0. 

Assumption A. 2. Each component gj^g^ of Ggg is an even, integrable 
function such that \gj,eoW\ < C\\f, A G 11, where 0</?<l, a - /3 < 1/2, 
j = l,...,r. 

Assumption A. 3. For each gj,0f^{\), j = 1, . . . ,r, one of the following is 
fulfilled: 

Condition 1. gj^Q^^ is Lipschitz of order greater than 1/2 on [0, vr]. 

Condition 2. gj^e^^ is continuous on IT and \dgj^eoW/d\\ < C|A|^^~^ for 
some < /3j < 1, 2a - j3j < 1. 

Condition 3. gj^o^ is of bounded variation on [0, vr] with finite disconti- 
nuities and a < 1/2, with \r{k)\ < Ck~^ for some t; > 1/2 (e.g., v =1 — a). 

Assumption A. 4. The r x r matrix Wq^ = /^(A)G6i(, (A)Gg^ (A) dA is 
positive definite. 

Assumption A. 5. On (0,7r], either (i) / is differentiable and |9/(A)/9A| < 
CIAI"*^"^ or (ii) each f{X)gj^gg{X) is of bounded variation or is piecewise 
Lipschitz of order greater than 1/2 on [0,7r], j = 1, . . . ,r. As n^oo, P(0 G 
ch° {TrG0g{Xj)[In{Xj) — f{Xj)]}jLi) 1, where ch°j4 denotes the interior con- 
vex hull of a finite set j4 C M*" . 

We briefiy discuss the assumptions. The bound on / in Assumption A.l 
allows for the process {Xt} to exhibit both SRD and LRD and is a slight 
generalization of (1). The behavior of Gg^ in Assumption A. 2 controls the 
growth rate of the scaled periodogram ordinates, GQg{Xj)In{Xj), at low fre- 
quencies under LRD and ensures that Wg^ is finite. Important processes 
are permissible under A.l and for these, useful estimating functions often 
satisfy A. 2. Assumption A. 3 outlines smoothness criteria for the estimating 
functions. The estimating functions treated by the FDB in [11] satisfy A. 3, 
including those for autocorrelations and normalized spectral distribution in 
Section 2.1. The functions f^^ and df^^ /dO from Examples 3 and 4 satisfy 
A. 3 for use in Whittle-like estimation and goodness-of-fit testing with many 
SRD and LRD models in the FDEL framework. For example, Hannan [20] 
considers Whittle estimation for ARMA densities for which functions in 
(11) satisfy Condition 1. The functions fg^ and dfg^/dO associated with 
the fractional Gaussian and FARIMA LRD densities in (3) and (4) fulfill 
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Condition 2 [10, 15, 17]. Process dependence that is not extremely strong, 
so that is integrable, allows greater flexibility in choosing more general 
estimating functions in Condition 3. 

For EL inference exclusively with the model-based functions L„.jr or Rn,T 
from Section 3.1, we introduce the additional assumption A. 5 which is gener- 
ally not restrictive. The probabilistic condition in A. 5 implies only that the 
EL ratio Rn,j^ can be finitely computed at 6q, resembling EL assumptions 
from [31] and [35]. 

4.2. Asymptotic distribution of empirical likelihood ratio and confidence 
regions. We now establish a nonparametric recasting of Wilks' theorem [43] 
for FDEL ratios under SRD and LRD, useful for setting confidence regions 
and making simple hypothesis tests, as in [33, 34, 35]. Define two scaled 
log-profile FDEL ratio statistics 

(17) 4W = -41ogi?„(0) and = -21ogi?„,^(0), 

using (15) and (16). The difference in the scalar adjustments to log-likelihoods 
in (17) is due to the assumption that the periodogram ordinates are "mean- 
corrected" in the construction of £„,:p(^)- In the following theorem, de- 
notes a chi-square distribution with v degrees of freedom: 

Theorem 1. Suppose Assumptions A.1-A.4 hold. If M = ^^W, then 
as CO, 

(i) in{eo)^xl 

(ii) Additionally, if A.5 holds and f = foa, then £n,T{Go) — ^ Xr- 

(iii) If Kji^e = 0, statement (ii) remains valid, even if M ^ Q . 

Remark 1. For a Gaussian {Xt} process, the fourth order innovation 
cumulant K4^£ = 0. 

Due to the data transformation aimed at weakening the time dependence 
structure, FDEL ratios closely resemble EL ratios with i.i.d. data [34, 38]. 
The formulation of estimating equations satisfying = in (6) is generally 
necessary for ^n(^o) to have a chi-square limit and is a consequence of this 
EL based on the periodogram. A similar moment restriction is shared by 
the FDB, as detailed in [11] (page 1938), due to difficulties in estimating 
the variance of empirical spectral means. Similar complications arise in the 
inner mechanics of FDEL, requiring = 0. As the proof of Theorem 1 
shows [see (24) and (27)], variance estimators intrinsic to FDEL ratios are 
asymptotically of the form given in Lemma 7 of Section 9 (i.e., setting gh = 
GG' there) and these target the asymptotic variance V of an empirical 
spectral mean appearing in Lemma 6 so that variance estimation within 
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FDEL is consistent if M = (or if the innovation cumulant n^^e = 0); see [32] 
for details. However, Section 2.1 gives some important estimating equations 
for which = and, importantly, estimating functions may be chosen with 
more flexibility for inference on Gaussian processes. 

If the true density / belongs to in (7), then a confidence region can be 
calibrated with ln,T{&) as well. Monti [31] suggests similar confidence regions 
for Whittle estimation with SRD linear processes and spectral densities pa- 
rameterized as in (9) (e.g., ARMA models). However, if the candidate density 
class J- is incorrect, confidence regions based on in,ri0) become conserva- 
tive to a degree dependent on the misspecification. This closely parallels the 
behavior of the EL ratio with misspecified regression models, as described in 
Section 5.4 of [35]. Confidence regions set with in do not generally require 
specification of a model density class but for the case of inference on 
Whittle parameters where a class of densities J-' may be involved, Section 6 
describes how (unlike in,^) ™ay be used, even if is misspecified. 

5. Extensions to maximum empirical likelihood estimation. We shall re- 
fer to the maximum of Rn{G) from (15) as the maximum empirical likelihood 
estimator (MELE) and denote it by 9n', we denote the maximum of Rn,j^{0) 
from (16) by 6n,j^- We next show that with both SRD and LRD linear time 
series, maximum empirical likelihood estimates (MELE's) On and 9n,j^ have 
properties resembling those available in EL frameworks involving indepen- 
dent data. 

5.1. Consistency and asymptotic normality. We first consider establish- 
ing the existence, consistency and asymptotic normality of a sequence of 
local maximums of FDEL functions Rn{0) and Rn.j^{0), along the lines of 
the classical arguments of [7]. The assumptions involved are very mild and 
have the advantage of being typically simple to verify and apply to both 
versions of FDEL Rn{0) and Rn,j^{9). Qin and Lawless [38] adopt a similar 
approach to study MELE's in the i.i.d. data scenario. 

Let II • II denote the Euclidean norm. For n £ N, define the open neighbor- 
hood Bn = {eeQ:\\e- OqW < n^''}, where r/ = max{l/3, 1/4 + (a - /3)/2, 
(1 + a -|- (5)/4} < 1/2 for 5 < 1 defined below in Theorem 2 and a, (3 from 
Assumptions A.1-A.2. 

Theorem 2. Assume A.1-A.4 hold and M = 0. Suppose, in a neighbor- 
hood ofOo, dGe{X)/d9, d'^Ge{X)/d9d9' are continuous in9 and \\dGg{X)/d9\\, 
\\d'^Ge{X)/d9d9'\\ are bounded by CIAI"*^ for some 5 <1, 5 + a<l. Suppose 
further that dGQf^/d9 is Riemann integrable and that Dg^ = /(A)9GeQ(A)/ 
dOdX has full column rank p. 
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(i) As n oo, there exists a sequence of statistics {9n} such that P 
is a maximum of Rn{0) and 9n G Bn) 1 and 




AT 



Ue, 



where Ve, = 4TT{D'g^^Wg~' De,y^ and Ue, = 7T~^Wg~\lrxr - {4tt)-^ De.Veo x 

(ii) Additionally, suppose A. 5 and f = fo^ hold, that dfe^/dO is Rie- 
mann integrable, that P{0 G ch.°{TrG g{Xj)[In{\j) — fei^j)]}jLi,d G Bn) — > 1 
for the closure Bn, and that in a neighborhood of 9q, Jq Gg{X)fg{X) dX = Ai 
and \\dfe/de\\, Wd"^ fe{X)/d9de'\\ < C|A|-", A G (0,7r]. Then there exists a se- 
quence of statistics {On,y^} such that P (On.j^ is a maximum of Rn,j^{0) and 
(^n,T £ Bn) — > 1 as n —> oo and the distributional result in (i) is valid for 

(iii) If i<-A,e = ^, then Theorem 2{\\) holds, even if AA ^ Q . 

Remark 2. When assuming / G in Theorem 2(ii), a constant function 
Jo Gg{X)fg{X) dX = Ai oi 6 represents a natural relationship between the 
chosen estimating functions and [e.g., the Whittle estimating equations 
in (11)]. The probabilistic assumption on the closure Bn in Theorem 2(ii) 
is similar to Assumption (A. 5) and implies the FDEL ratio in,T{G) exists 
finitely in a neighborhood of ^o- 

As pointed out by a referee, Theorem 2 establishes consistency of a local 
maximizer of the EL function only. In the event that the likelihood Rn or 
Rn,j^ has a single maximum with probability approaching 1 [e.g., by concav- 
ity of Rn{0)], the sequence {9n} or {On^j^} corresponds to a global MELE. 
In many cases, the consistency of global maximizers can also be established 
using additional conditions. In Theorem 3 below, we give conditions for the 
consistency of On] similar conditions for 6n,j^ can be developed. Note that 
these conditions are satisfied by the estimating functions given in Section 2. 

Theorem 3. Suppose Assumption A.l holds. 

(i) Assume 9o lies in the interior of Q, Gg{X) is a (componentwise) 
continuous and monotone function of for A G 11 and, for 9 |G5i(A)| is 
Riemann integrable and bounded by C|A|~'' for some 6 <1, a + 6 <1. Then 
as CO, 



(18) P(^^„ = argmaxi?„(^) exists j — >1 and On — > Oq. 

(ii) Suppose G is compact, Ge{X) = G^ from (11) [or Gg{X) = G^*(A) 
from (12)] is continuous at all (A,0) G II x © and W{9) from (8) attains its 
minimum on the interior of Q. Then (18) holds as n ^ oo. 
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5.2. Empirical likelihood tests of hypotheses. EL ratio test statistics with 
and 9n 

are possible for both parameter and moment hypotheses. Sim- 
ilarly to parametric likelihood, we can use the log-EL ratio in{do) — ^n(^n) 
to test the parameter assumption Hq -.O = 6q. For testing the null hypothesis 
that the true parameter satisfies the spectral mean condition in (6), the 
log-ratio statistic £n{On) is useful. Analogous tests are possible with ln,r{Oo) 
and in,r{(^n,j^) ■ These EL log-ratio statistics have limiting chi-square distri- 
butions for testing the above hypotheses. 

Theorem 4. Under the assumptions of Theorem 2 with the sequences 
{9n} and {6'„,jr}, 

(i) iniGo) — £n{dn) Xpy £n{Sn) Xr~p ^'^^ these are asymptotically 
independent; 

(ii) in,rido)-£n,Tidn,y^) ^ Xp> 4,^(^n,jf) Xr-p ".nd thcsc are asymp- 
totically independent, if the assumptions in Theorem 2(ii) are additionally 
satisfied; 

(iii) if Ki^e = 0, Theorem 4(ii) remains valid, even if M ^ . 

In Sections 6 and 7, Theorems 1-4 will be applied to Whittle estimation 
and goodness-of-fit testing in the FDEL framework. 

5.2.1. Parameter restrictions and nuisance parameters. Qin and Lawless 
[39] introduced constrained EL inference for independent samples and [23] 
provided a blockwise version for time domain EL under SRD. We can also 
consider FDEL estimation subject to the following system of parameter 
constraints on 9 : iIj{9) = E W^, where q <p and ^^{9) = dTp{9)/d9 is of full 
row rank q. By maximizing the EL functions in (15) or (16) under the above 
restrictions, we find constrained MELE's 9^ or 9^ -p. 

Corollary 1. Suppose the conditions in Theorem 2 hold and, in a 
neighborhood of 9q, ip{9) is continuously difjerentiable, \\d'^ip{9)/d9d9'\\ is 

bounded and ^'(^o) is rank q. If Hq: V'(^o) = holds, then ln{9t.) —in{9n) 

Xg and ini9o) - (■n{9t) xl-q as n^oo. 

We can then sequentially test Hq:'iP{9q) =0 with a log-likelihood ratio 
statistic £ni9t) - £n{9n) and, if failing to reject Hq, make an approximate 
100(1 — 7)% confidence region for constrained 9 values {9 : 1^(9) = 0, in{9) — 

Uot)<xl-,,i-,}- 

Profile FDEL ratio statistics can also be developed to conduct tests in the 
presence of nuisance parameters (see Corollary 5 of [38] for the i.i.d. case). 
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Suppose 6 = {61,62)' , where 61 and 62 are qxl and {p — q) x 1 vectors, 
respectively. For fixed 61 = 6^, suppose that 62 and ^2 maximize the EL 
functions Rn{6i,62) and Rn,r{(^iT^2) with respect to 6*2- 

Corollary 2. Under the conditions in Theorem 2, if Hq -.61 = 61 holds, 
then i^(6ld^2)-^ni0n)^xl as 00. 

If the assumptions in Theorem 2(ii) are also satisfied, Corollaries 1 and 2 
hold using ^n,y^, ^t^T ^2,T- 

6. Whittle estimation. 

Example 4 (Continued). With SRD linear processes, Monti [31] sug- 
gested EL confidence regions for Whittle-like estimation of parameters 6 = 
icP' characterizing fe^T from (9). Theorem 1 provides two refinements 
of [31]. 

Refinement 1. Monti [31] develops an EL function for 6 by treating the 
standardized ordinates In{\j) / fei^j)-, 3 = l,---,-^, as approximately i.i.d. 
random variables, similarly to the FDB. The EL ratio in (4.1) of [31] asymp- 
totically corresponds to ln,r{(^) when using the estimating functions Gq from 
(11). With this choice of functions, the nonzero spectral mean M^] 7^ in 
(11) is due to the first estimating function fg^ intended to prescribe o"^. Note 
that the use of ln,T{(^) and Gg for setting confidence regions requires the 
additional assumption that the fourth order innovation cumulant K^^e = 0, 
by Theorem 1. Valid joint confidence regions are otherwise not possible here 
because A4w 0- This complication due to u^-inference is related to the 
inconsistency of the FDB Whittle estimate of o"^ when k^^^ 7^ 0, as described 
by [11]. While valid for SRD Gaussian series with n^^^ = 0, periodogram- 
based EL formulation in [31] may not be applicable to general SRD linear 
processes. 

Refinement 2. Treating o"^ as a nuisance parameter and concentrating it 
out of the Whittle likelihood, Monti [31] suggests an EL ratio statistic for 
estimation of the remaining p — 1 parameters 'd in (9) via confidence re- 
gions. The statistic (6.1) of [31] appears to be asymptotically equivalent 
to —2 log /?„(■!?) = 1/2 • ln{'&)-, based on the p — \ estimating functions G^* 
from (12) and the = [(n — 1) /2j periodogram ordinates. Note that for the 
function GJj*, we have Mw* = so that Theorem l(i) applies to ln{'&)- 

For Whittle-like estimation of in the parameterization from (9), the 
EL log-ratio ln{'&) based on the functions G^* in (12) appears preferable to 
tn^jr{'d). This selection results in asymptotically correct confidence regions 
for t? under both SRD and LRD, even for misspecified situations (/ J^) 
where the moments in (10) still hold. 
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7. Goodness-of-fit tests. 

7.1. Simple hypothesis case. 

Example 3 (Continued). We return to the simple hypothesis test Hq : f = 
fo for some possible density /q. To assess the goodness-of-fit, Milhoj [30] and 
Beran [3] proposed the test statistic r„ = irAn/ for mixing SRD linear 
processes (with k^^^ = 0) and long-memory Gaussian processes, respectively, 
and established the limiting bivariate normal law of ^/n{{An, Bn)' — {2tt, tt)'} 
under Hq for A„ = 27r/nEf=i I^{Xj)/ f^{Xj), B^ = 27r/nEf=i /n(A,)//o(A,). 
Since these linear processes involve ^4^^ = 0, under Theorem l(iii), we can 
construct a single statistic in,T to test Hq by treating = {/o} in (7) and 
employing a single estimating function satisfying (6) with A4 = 11 under 
Hq. In expanding in,T = ^{Bn — vr)^/ {it An) + Op(l), we find the FDEL ratio 
statistic asymptotically incorporates much of the same information in T„ 
under Hq (with better power when / = c/o, c 7^ 1). 

For testing the special hypothesis HQ:{Xt} is white noise (constant /), 
a FDEL goodness-of-fit test based on the process autocorrelations can be 
applied, similar to some Portmanteau tests [27, 28]. The estimating functions 
G(A) = (cos(A), . . . ,cos(mA))' satisfy (6) with M = OeM.'^ under this Hq 
(see Example 1) and yield a single EL ratio In which pools information 
across m EL-estimated autocorrelation lags. 

7.2. Composite hypothesis case. To test the composite hypothesis Hq : f £ 
for a specified parametric class various frequency domain tests have been 
proposed by [3, 30, 37] which use Whittle estimation to select the "best" 
fitting model in and then compare the fitted density to the periodogram 
across all ordinates. We show that FDEL techniques can produce similar 
goodness-of-fit tests, while expanding our EL theory slightly. 

Suppose {Xt} is a Gaussian time series and that we wish to test if f € 
for some parametric family as in (9), which includes densities (3) or (4). This 
scenario is considered by [3] and [37] for LRD and SRD Gaussian models, 
respectively. FDEL methods may simultaneously incorporate both compo- 
nents of model fitting and model comparison through estimating equations 

(19) rG^fdX = M^, rif/fefdX = 7r, 

Jo Jo 

where Gg = {fg^,dfg^/d'0'y are the Whittle estimating functions from (11) 
for the parameters 9 = ((T^,t?')' G in Note that we introduce an overi- 
dentifying moment restriction on / in (19) so that r =p+l. We then extend 
the log-likelihood statistic in,^ in (17) to include ordinates by defining 
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^,2,^(0) = -2\ogRj2^r{0) for Rj.^:f{0) = {N/^)^L,.^M and 

( N N 

LilA^) = sups n • ^ 0, X! = ^' 

f / l2(A,)/{2/2(A,)} - 1 
^tl HG^(A,)[/n(A,)-/,(A,)] 

using /g, /| above to approximate E(/„), E(/^/2) for each ordinate. 

To evaluate Hq : f £ we test if the moment conditions in (19) hold for 
some 6 value. Following the test prescribed by Theorem 4, we find the argu- 
ment maximum of Lj2 jr{6), say j,, and form a test statistic ■^/2^:r(^/2^) 
for this Hq. The subsequent extension of Theorem 4 gives the distribution 
for our test statistic. 

Theorem 5. Suppose {Xt} is Gaussian and the assumptions in The- 
orem 4(ii) hold for fg^ and Gf^^ with a — [3 <v for each arbitrarily small 

t; > 0. Under the null hypothesis f = feo ^ ~^ Xi as oo. 

The distributional result is valid even with nonzero spectral mean conditions 
in (19) because the process is Gaussian. We make a few comments about 
model misspecification. Suppose f ^ but still represents the parameter 
value which minimizes the asymptotic distance measure W{9) in (8) and fe^ 
satisfies (10) (i.e., f^ G'^^fdX = Ai^i holds). A test consistency property can 
then be established: as n ^ oo, 

where oq > depends on / and fe^^ ■ We are assured that the test statistic 
can determine if Hq : f £ is true as the sample size increases. 

8. Conclusions. We have introduced a frequency domain version of em- 
pirical likelihood (FDEL) based on the periodogram, which allows spectral 
inference, in a variety of applications, on both short- and long-range depen- 
dent linear processes. Further numerical study and development of FDEL 
will be considered in future communications. See [32] for extensions of FDEL 
using the tapered periodogram in (13) under weak dependence. A valuable 
area of potential research includes Bartlett corrections to the EL ratios in 
(17) (see [12] for i.i.d. data). Second- and higher-order correct FDEL con- 
fidence regions may be possible without the kernel estimation or stringent 
moment assumptions required with the frequency domain bootstrap of [11]. 
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9. Proofs. We only outline some proofs here for reasons of brevity. De- 
tailed proofs can be found in [32] and we shall refer to relevant results given 
there. We require some additional notation and lemmas to facilitate the 
proofs. In the following, C or C(-) will denote generic constants that depend 
on their arguments (if any), but do not depend on n, including ordinates 

Define the mean-corrected discrete Fourier transforms dnc(A) = J2t=ii-^t — 
/i)e-**^, A G n. Note that 27m/nc(A) = \dnciXT = dnc{X)dnc{-X) and Inc{\j) = 
In{Xj) for j = l,...,N. Let Hn{\) = YJl=i e"**^, A G M, and write K„(A) = 
(27rn)~^|i7„(A)p to denote the Fejer kernel. The function Kn is nonnegative 
and even with period 2-k on M and satisfies /q Kn dX = l (see page 71 of [6]). 
We adopt the standard that an even function g : 11 — > R can be periodically 
(period 27r) extended to M by g{\) = g{-X), g{\) = g{\ + 2tt) for A G M. We 
make extensive use of the following function from [8]. Let Lns ■ R — > R be 
the periodic extension of 

{e~*n, |A| < e*/n, 

^^J^, eVn<|A|<vr, ^ G H, . = 0, L 
|A| 

For each n > 1, s = 0, 1, Lns{-) is decreasing on [0, vr] and 

i7r?7 

(20) \Hn{\)\<CLno{\). Lno(A)<— ^ Tl^, A G M. 

1 -|- |Amod27r|n 

In the following, cum(Yi, . . . , Y^) denotes the joint cumulant of random 
variables Yi, . . . ,1^. We often refer to cumulant properties from Section 2.3 
of [5], including the product theorem for cumulants (cf. Theorem 2.3.2). 

We remark that Lemma 5 to follow ensures that the log-likelihood ra- 
tio ^n(^o) exists asymptotically. Lemmas 6 and 7 establish important re- 
sults for Riemann integrals based on the periodogram under both LRD and 
SRD; Lemma 6 considers the distribution of empirical spectral means and 
Lemma 7 is required for variance estimation. 

Lemma 1. Let 1 < i < j < N and < d < 1. If a-i e {±Aj}, aj G {±Aj} 
and + aj / 0, then 



LnoicLi + aj) < 



nCijn 



(i) (U-i) , sign(ai) /sign(aj), 

Cijn=< (i + i)"\ sign(aj) =sign(aj), i + j <7i/2, 

[(n-j-i)"\ sign(ai) =sign(aj), i + j>n/2. 

(ii) Lniitti + aj) < min{log(n7r)L„o(ai + Uj), nC{d)[cijn]'^}- 
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(iii) / L™o(A)(iA<C(m)[log(n)+n'"-^], m>lG 



LZ{ri + X)LZ{r2-X)dX 

n 

(iv) ( CLni{ri + r2), m = l, 

<< ri,r2GM. 
[CnLloin+ri), m = 2, 

Proof. Parts (iii)-(iv) are from Lemmas 1 and 2 of [8]. Lemma l(i) 
follows from the fact that \{ai + aj)mod27r| > In/n if Oj + aj ^ 0, along 
with the definition of L„o- D 

Lemma 2. Suppose Assumption A.l holds. Let lip = [p, vr] /or < p < 
TT. // ai,a2 G n, |ai| < \a2\ and \a2\ G Hp, i/ien cum((i„c(ai), d„c(a2)) = 
27ri7n(ai + 02)7(02) + -Rnp(ai, 02) and i?np = o(n) /loWs /or i^np = sup{|it:np(ai, 02) 
ai, 02 S n, |ai| < 1 02 1, 1 02 1 G Hp}. 

Proof. We modify the proof of Theorem 1(a) of [8]; see [32], Lemma 5 
for details. □ 

Lemma 3. Let I <i < j < N {n>3) and oi, . . . , Ofc € {±Ai, ±Aj}, |ai| < 
I02I < • • • < \ak\, with 2 <k <8. Under Assumption A.l, 

\cum{dnc{ai),dnc{a2))\ 
(i) ^ ^ (\ai\~''{\a2\~'^ + Lni{ai+ 02)), if a > 0, 

- \L„i(ai + 02), if a = 0. 

|cum((i„c(ai), • • . ,dnc(afc))| 

k 

< C{|a,r/2-i|afc„iri/2 + nlog'^-Hn)} J] W.^^'- 

i=i 

Proof. We show Lemma 3(ii); the proof of (i) can be similarly shown 
with details given in [32], Lemma 1. From [25, 41], the kth order joint cumu- 
lant {2 < k < 8) may be expressed as cum{dnc{ai) , . . . , dnc{ak)) = 
(27r)~'^+^Ke^fci/(n'^~^) using the kth order innovation cumulant K^^k and a 
function v of Borel measurable sets A C defined as 

k k-1 \ /k-1 \ 



fe-i 

X Y[{Hn{zj)b{zj - aj)}dzi . ..dzk-i. 
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OnB = n'^ZlKzi, . . ■,Zk-i) G n'^-^ : \zj - aj\ < \aj\/2k}, we have \Hn{zj)\ < 

C\aj\-\ \Hn{Ej=iaj-EjZl < C^lflfcl"' by (20) and, by applying Holder's 
inequality. 



l^fe-i-°fe-i 

<lafc_il/2fe 



'k~l 



dzk-i<C\ak-i\^^- 



'")/2 



while Ji,^_a,\<\a,\/2k \Kzj - aj)\dzj < C\aj\^ "/^ for 1 < j < A; - 2. By these 
inequalities, < C\ak\~^\ak^i\~^/^ll^jZl\aj\~°'/'^ . Now, on Bj = 

{{zi, . . . ,Zk-i) G n*^""^ : \zj — Ojl > |aj|/2/c} for fixed 1 < i < A; — 1, we find 
\b{zj - aj)\ < C|aj|""/2, < Cn and 



Hn[ ^CLi-^Zijbi ^ai- Zi 
1=1 / \e=i / 

C|afcr^|Ar"/2 dA + 



dz^ 



< 



C|ofc|-/Xo(afc-A)dA 



|A|>|afe|/2 



|A|<|afe|/2 

<C\ak\~''/^ login), 

by Lemma l(iii), while JYi\Hn{z£)\\b{ze — a£)\ dzi < C |a£|~°/^ log(n) for each 
e^j. Since H^-^ \B = -Bj, we have \u{U'''^\B)\ < ZjZl WiBj)\ < 



Cn log 



k-l. 



(n)n,ti|a,|-"/2. □ 



Lemma 4. Under Assumption A.l, f{k) = r{—k) = n ^ J2]=i{-^j ~ A*) ^ 
{Xj^k — lA r{k) = Cov{Xj, Xj^k) as n^ CO for each k>0. 

Proof. f{k) is asymptotically unbiased and Var(f(A;)) = o(l), by Lemma 3.3 
of [22]. □ 

Lemma 5. Under Assumption A.l, suppose G= {gi, ■ ■ ■ , Qr)' = Gg^^ is 
even with finite discontinuities on [0,7r] and satisfies Assumptions A. 2 and 
A.4. // /nG/dA = e W, then P{0 G ch.°{7rG{Xj)In{Xj)}f=i) ^1 as n ^ 
oo. 



Proof. See [32] for details. It can be shown that inf yj^n^^ /-q l{y G(A)>o}y' ^ 

GfdX>C>0 and that P(inf f Ef=i /„(A,)y'G(A,)l{yG(A,)>o} > C/2) 
— > 1 follows using Lemma 4 with arguments from Lemma 1 of [20]. When 
the event in this probability statement holds, the separating/supporting hy- 
perplane theorem implies that G ch°{7rG(Aj)/„(Aj)}^]^. □ 
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Lemma 6. Suppose Assumptions A.1-A.3 hold with respect to an even 
function G = {gi, . . . ,gr)' = Gg^ and let Jn = {271 /n) Y^f=i G{Xj)In{)^j) ■ Then 

V^{Jn — Jq fGdX) AA(0, V) as oo, where 

If, in addition, A. 5 holds, then \/nJn — ^ AA(0, V) for Jn = {2-K/n) J2jLi x 
[I„(A,)-/(A,0]. 

Proof. By Assumptions A.1-A.2, we have that ^/n{f^ GIncdX - 

E GIncdX) — ^ N{0,V) from Theorem 2 and Lemma 6 of [17] and the 
Cramer-Wold device. To show the result for J„ in Lemma 6, it suffices to 
estabhsh that 



(21) 



E / G{In 




f)d\ 



GInr. dX 



o{n 



Op{n 



-1/2^1 



The distribution of Jn then also follows, using || (27r/n) X^jLi G'(Aj)/(Aj) — 
/(7/GdA|| =o(n-i/2), which can be established with straightforward argu- 
ments; see [32], Lemma 10. 

Without loss of generality, we assume that G = g (i.e., r = 1) and es- 
tablish (21) under Condition 2 of Assumption A. 3; see [32] for proofs un- 
der Conditions 1 or 3 of A. 3. Proving (21) under Condition 1 of A. 3 in- 
volves using the nth Cesaro mean Cng{X) = JjjKn{X — y)g{y)dy, A G H, for 
which sup;^gn Idi-^) ~ Cn9(A)| = o(n~^/^), by Theorem 6.5.3 of [14], and us- 
ing ^J2]j=-NCng{Xj)Inc{Xj) = lYiCnglncdX, E Jj^glncdX = JjjCngf dX. The 

proof under Condition 3 relies on Theorem 3.2 and Lemma 3.1 of [9] and 
Lemma 4 of [8]. 

We show the first convergence in (21) here; Lemma A.l in the Appendix 
gives the second part of (21). Using the evenness of Kn, Jjj Kn{X — y)dy = l 
and E(/„c(A)) = Kn{X - y)f{y) dy, we have 



E / gIncdX - [ gfdX 
Ju Jn 



<^ Kn{X-y)f{y)\g{X)-g{y)\dydX 
Ju^ 



(22) 



(0,7r]: 



fiyMX)-g{y)\ 
(l + |A-y|n)2 



dy dX, 



where the last inequality follows from (20) and | (A — ?/)mod27r| > 1 1 A| — |?/| | , A, 
y E n. We now modify an argument from [17] (page 99). Under Condition 2 
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of A. 3 (with respect to Pi > 0), we may pick < 7 < 1/2 so that < 7* = 
7 + - 7) - a < 1 and fivMX) - g{y)\ < C(min{y, A})"1+t* |A - y\^-"' , 
A, y G (0,7r]. We then bound (22) by 

using the fact that there exist a C > such that J()° y~^~^^ {l + \X — y\)~^~'^ dy < 
C|A|-i+7* for any A G M. □ 

Lemma 7. Under Assumption A.l, suppose g and h are real-valued, 
even Riemann integrable functions on U such that \g{X)\, \h{X)\ < C|A|^, 
0</3<l, a — (3 < 1/2. Then as 00, 

— J2g(X,)hiXj)l'^iX,) and 2.—Y,g{X,MXj)iUX,)- fiXj)f 

ghfdX. 

Proof. We consider the first Riemann sum above; convergence of the 
second sum can be similarly established. Since (27r/?i) J2jLi 9i-^j)h{Xj)f'^{Xj) 
Jo 9^f^ dX by the Lebesgue dominated convergence theorem, it suffices to 
establish 



Bn 



47r ^ 



HSn)-—Y.9iXj)hiX,)f\Xj) 



n . , 



0(1), Var(5n) = o(l), 



for Sn = (27r/n) J2j=i 5(Ai)/i(Aj)I^(Aj). We show that S„ = o(l); Lemma A.2 
in the Appendix shows that Var(5.„) = o(l). By E((i„c(A)) = and the prod- 
uct theorem for cumulants, 

(27rn)2E(/2(Aj)) = cum^{dnc{Xj),dnc{Xj)) + 2 curn^ {dnc{Xj),dnc{-Xj)) 

+ CUm{dnc{Xj),dnciXj),dnc{-Xj),dnc{-Xj)). 

(23) 

Then i?„ < Bin + ^2n + B^n ^or terms Bin defined in the following. 

Using the fact that n"^27rL„o(2Aj) < (2j)~H{j<L„,/4j} + (n- 2j)~il{j>L„/4j}, 

by Lemma 1, and the fact that cum{dnc{Xj) , dndXj)) < CXj°'{Xj^ + 
log(n)L„o(2Aj)), by Lemma 3(i), we have, for Bin = ^'^^2^=1 l5'('^j)^('^i)l ^ 
cum'^{dnc{Xj),dnc{Xj)), that 

[n/4j TV 

(n-2jr' I =0(1). 

j=[n/4j+l 

By Lemma 3(ii), \cum{dnc{Xj) , dnc{Xj) , dnd- Xj) , dnc{- Xj))\ < Cn{n^/'^ + 
log^(n))Aj^" so that B2n = n^^J2jLi\9{^j)H^j)c^Mdnc{Xj),dnc{Xj), 



I Ln/4J TV 

V 7=1 _, = |n/4|+l 
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dnc(— Aj), dnc(— = 0(1). Choose < p < vr. Using Lemma 3(i) and Lem- 
ma 2, for an arbitrarily small p, we may bound 

iEis3n<in^f- E Af-2° + ^.^ E xf- 



Xi<Xj<p 



AT 



for = \4:TT / nY,jLi 9{^j)h{Xj)[cumj — /(Aj)][cumj + /(Aj)]|, where we de- 
note cumj = cum{dnc{Xj) , dnc{— Xj)) / {27rn) . As C does not depend on p 
above, B^^ = o(l) follows. □ 

Lemma 8. Suppose Assumption A.l holds and < (3 <1, a — (3 < 1/2. 
Lei u; = max{l/3, 1/4 + (a — /5)/2}. T/ien maxi<j<Ar /„(Aj)A^ = Op(n'^) and 
maxi<j<Ar /(Aj)A^ = o(n'^). 



Proof. It holds that E(l4(A,)) = cum(/2^(Aj),/2^(A,)) + [E(j2,(A,))]^ 



< 



CAj- by the product theorem for cumulants; see (33), (23) and Lemma 3. 
For each e > 0, we then have 



P( max^I„(A,)Af > en^) < ^ ( E Af E(4,(A,)) 



1/4 



which follows from n X^^i ^"^ = 0(1). □ 



Proof of Theorem 1. We give a detailed argument for Theorem l(i); 
parts (ii)-(iii) of Theorem 1 follow with some minor modifications. By Lemma 5, 
G ch°{7rG0,j (Aj)/„(Aj)}j^]^ C M'' with probability approaching 1 as n ^ 00 
so that a positive Rn{0o) exists in probability. In view of (15), we can express 
the extrema i2„(6'o) = 11^^1(1 + 7i)~"^ with 7^ = t'Q^7rGeg{Xj)In{Xj), hj\ < 1, 
where tg^ £ W satisfies Qin(,Go,tg^^) = for the function Qin(-, •) on x 
defined in (28). Let 

Wn0o = — '^Geo{Xj)G'g^{Xj)In{Xj), 

fl ^ 



(24) 



2vr.^ 



JnOo = Qln{8o,0) = E ^6>o(Aj,)/n(Aj). 



n . 



By Lemma 6 with Jjj Gg^ f dX = A4 = and Lemma 7, we have 
(25) |J„eol=Op(n-i/2), \\Wneo-We,\\ = Op{l) 
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so that WnOo is nonsingular in probability. Using Assumption A. 2 and Lemma 8, 
it holds that 

(26)y„= max 7r||Geo(A,-)||/n(A,) = Op(n^/'), p,J| = Op(n-V2), 

l<j<A' 

where the order of Ht^Q || follows as in [33, 34]. Note that by (26), maxi<j<Ar \ < 
p6»oll^n = Op{n~'^/'^)op{n^^'^) = Op{l) holds. Algebraically, we write 
= Qinieo,te,) = Jneo - TrWneoteo + (W^) Ef=i Ge,iXj)In{Xjhj /{l + 7i) 
and solve for te^ = {TrWneo)'^ Jneo + ^Pn, where 



K\\<Yn\\teo\ 



N 



\^nol\\ E l|G,„(A,)f 4(A,)j { max (1 +7,)-^} 



by Lemma 7, (26) and the fact that maxi<j<Ar \ = Op{l). When p6ig||y„ < 
1, we apply a Taylor expansion log(l + 7j) = jj — 7? /2 + Aj for each 1 < 
j <N. Then 



N 



(27) 



4(0o) = 4^1og(l+7,) = 2 



N 



N 



N 

+ 4E^-' 



AT 



N 



2E7.-E7I 

. i=i i=i . 



By Lemma 6 and (25), nJ^Q^{7rWn0o) ^JnOo Xr- We also have ^^</''„(v^W„e(,) x 



(t>n = Op(l) and we may bound J2j=i l^jl by 
(1 - WteoWyn)-^ n p'^ 

= Op(l), 

from Lemma 7 and (26). Theorem l(i) follows by Slutsky's Theorem. 

Under Theorem l(ii)-(iii), fg^ = f holds and Rn,j^{do) exists in probability 
as n — > 00, by Assumption A. 5. We repeat the same arguments as above, 
replacing each occurrence of /n(Aj) with /n(Aj) — /(Aj) instead; we denote 
the resulting quantities with a tilde: 

N 



^neo = ^J2Geo{Xj)G'e,{Xj)iIn{Xj) - /(A,))^ 

N 



27r 



JnOo = — E Geo{Xj){In{Xj) - f{Xj)), 
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7j, Aj, (pn, etc. All the previous points follow except for two, which are 
straightforward to remedy: by Lemma 7, ||2W„eo ~ ^^oll ~ '^pi^) instead of 
(25), and in (27), we must write 

N 

^n,^(eo) = 2^1og(l + 7,) 
i=i 

TV 

i=i 

where nJ!^Q^{2TTWneo)''^ JnOo Xr by Lemma 6, n(f>'^{2-^TTWneo)4>n = Op{l) 
andEf=iAj = Op(l). □ 



Proof of Theorem 2. We require some additional notation. Define 
the following functions on x M*": 

27r^ Ge{Xj)In{Xj] 



n (ft f) - \^ ^ev^jj^nyyjj 



(28) 

n ^^ '^^^ In{W{dGe{\,)/de)'t 

Also, define versions Qinid,t) and Q2n{(^-,t) by replacing each In(Xi) with 
In{Xi) — fe{Xi) in (28) and adding the extra term —{dfg{Xi)/d9)G'g{Xi)t to 
the numerator of Q2n- We use the following MELE existence result to prove 
Theorem 2; see [32] for its proof: 

Lemma 9. Under the assumptions of Theorem 2, 

(i) the probability that Rn{0) attains a maximum Orn which lies in the 
ball Bn and satisfies Qin{On,tg ) = and dln{0) / d9\g^g =2nQ2n{Gn,t§ ) = 
0, converges to 1 as n—> oo; 

(ii) under the assumptions of Theorem 2(ii) or (iii), result (i) above holds 
for Rn,j^{0) with respect to 6n,J^, £n,r, Qin{6n,r,tg^ 2-^Q2n{9n,r-,tQ^ ^) 

[replacing On, in, Qln{On,tgJ, Q2niOn,tgJ]. 

We now establish the asymptotic normality of On, following arguments in 
[38]. Expanding Qinidn,tg ), Q2n{0n,tQ ) at (^OiO) with Lemma 9, we have 
that 





— JnOo + Ein 


\0n — OoJ 


E2n 
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9Qin(eo,o)/at dQin{o^,^)/de 
dQ2n{eo,o)/dt 



dQin{Oo,0) 



dt 



-ttW, 



dQ2ni0o,0) 

dt 



dQini9o,0) 



86 



dQ2ni0o,0) 

86 



0, 



with Jneo^^neo as in (24). In addition, it can be shown that 



dQmiOo, 0) _ 27r ^ 9Ggo(A,) ^ , _ , 



86 



89 



using (27r/n) X]j=i /(Aj)c?G6ip (Aj)/96' —> 1)90/2 by the dominated conver- 
gence theorem and a modification of the proof of Lemma 7. Applying this 
convergence result with (25), 



(29) 







^ 1 " 






' 27r . 



holds. One iiiciy verify tliat ||-E'ir 



\E' 



2n\ 



Op(5„n-i-''Ef=i^, 



j=l ^ 
-25 



max{j2(Aj),4(Aj)}) = Op(5.„) for 5„ = ||t0^|| + ||6'„-6'o|| and '^jn = n "^X. 



+ ^j^- By Lemma 6, \/nJndQ AA(0, vrVF^o), and so it follows that 
Sn = Op{n~^/'^). We then have that 



.2/3 



(30) 



^/n{6„ 

y/E{tn 



0) 



-V^A2inJneo + Op{l) — >Af{0, Ve 



do)-, 



-^/nAllnJneo + Op(l) — >M{Q,U0a). 



To show the normality of On^jr — 9q under the conditions of Theorem 2(ii)- 
(ii), we substitute Qin,Q2n for Qin,Q2n above and, using Lemma 9, repeat 
the same expansion with an analogously defined matrix S„ (having com- 
ponents Aijn in S-i). Note that 8Qin{6o,0)/89 = 8Qin{6o,0)/86 - DnOo 
where, by the Lebesgue dominated convergence theorem, as n — > 00, 



27r ^8[Ge{X^)fe{Xi)] 
1=1 



89 



0=00 



ld_ 
286 



feGedX 



0, 



0=00 



since the theorem's conditions justify exchanging the order of differentia- 
tion/integration of fgGe at ^0 and J^i feGe dX = M. is constant in a neigh- 
borhood of ^0- The convergence result in (29) follows for S"^ upon replacing 
"-27rC/0o , 27ry0o" with ''-AirUe, , ttVo, ." By Lemma 6, ,/EJneo = VnQin{6o, 0) 
M{0,7rWeo) so that (30) holds for v^(^„,^ - 9o) and \f^{t§^^- 0)/2 after 

replacing J^gg, ^21n, ^lln with JnOo, ^21n, ^lln/2. □ 
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Proof of Theorem 3. To establish (i), assume without loss of general- 
ity that G0{X) is real-valued and increasing in 6 (i.e., r = 1). For 6 gQ, define 
En,e = F Ei=i{7r/n(A,)G9(Aj) -M}, where M = GgJdX. Following the 

proof of Lemma 7, Enfi Jo Gef d\ — M. holds for each 6 so that, for an 
arbitrarily small e > 0, P{Enfio-e < < — > 1 by the monotonicity of 
Gg. When the event in the probability statement holds, there exists 9n&Q 
where E_^^ =0, by the continuity of Gg, and Ln{9n) = (tt/N)^ follows 
in (13). For any 9 with \9 — 9q\ > e, we have that Enfi Oi implying that 
Ln{9) < Ln{9n). Hence, a global maximum 0„ satisfies P{\On — 9q\ < e) — > 1. 
For (ii), we consider Gq = Gf and Ai = A4w from (11). Suppose 6^ and 

9* denote the minimums of Wn{9) = ilogg + ^J2f=i^In{^j)fe\^j) 
and W{9), respectively. Using Lemma 4 with arguments as in Theorem 1 
of [20], it follows that 0* 9*. Since 9* is interior to 0, it holds that 
dWn{9n) / 99 = 0, which implies that En^^ = under the above definition 
of Enfi with G^ Hence, Ln{9^) = {tt/N)^ and, using Lemma 4 with 
arguments of Lemma 1 of [20] , = En^e^ Jq Gg* f dX — M holds, whereby 
9* = 9(), by uniqueness. We then have a global maximum 9n = 9^, for which 
9n 9o. □ 

Proof of Theorem 4. Considering Theorem 4(i), let Px = X{X'X)~'^X 
and Irxr denote the projection matrix for a given matrix X and the r x r 
identity matrix. Writing {TrWgg/n)^/'^Zng = Jno^ + DgQ{9 — 9o)/2, it holds 
that \il9)-Z'^QZng\=Op{l + n\\9-9of) uniformly for 9 £ Bn (see [32] , Theo- 
rem 3) sothat^„(^„) = Z'.{Irxr- Py^^-i/2r. )Zneo+Op{l), using (29)-(30). 

By (27), in{9o) = KiOo^neo + Op{l), where Z^g^, -^M{0,lrxr), by Lemma 6. 
Theorem 4 follows since P -1/2 , Lxr — -P^.-i/Zr, are orthogonal idem- 

potent matrices with ranks r, r — p. □ 

Proofs of Corollaries 1 and 2. Nordman [32] gives a detailed proof 
of Corollary 1 and Corollary 2 follows by modifying arguments from Corol- 
lary 5 of [38]. □ 

Proof of Theorem 5. Nordman [32] provides details where the most 
important, additional distributional results required are (27r/-^/n) ^jLi ^neoj 

AA(0,7rVF,J, (2vr/n)Ef=i^neojKeo,i ^ ^oj2 for Y^g,,, = (/^(A,)/ 
[2/2 (A,)] - 7r,/„,(A,)G^ fA,) - M^)' and 



'Wg\ 





> Wgl = 


'lOvr 


Att' 





Wg*:_ 




27r_ 
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2 "^JBq 

^0 d^i 



■d\ 



«J = lv,P-l 



The convergence results can be shown using arguments in [3] and [16]. □ 

APPENDIX 

Lemma A.l. Suppose Assumptions A.1-A.2 hold for a real-valued, even 
function g satisfying Condition 2 of Assumption A. 3. Define the nth Cesdro 
mean of the Fourier series of g as Cng{X) = /j-^ Kn{X — y)g{y) dy, A S 11, and 
let Cn = Jq Cng{X)Inc{X) dX. Then as oo, 



(31) 



giX)InciX)dX 



2tt 



N 



— ^9{Xj)In{Xj)-Cr. 



Op(l)> 



Op{l) 



Proof. Assume that g satisfies A.l and Condition 2 with respect to (3 
and respectively. We establish the first part of (31). Using /j^ Kn{X — y) dy 
= 1, E(I„c(A)) = J^iKniX — y)f{y)dy and Fubini's Theorem, we write 
2^ f^{cng - g)IncdX\ as 



E 



Inc{X) 



Kn{X-y)[g{y)-g{X)]dy 



dX 



< / Kn{X-z)KniX-y)fizMy)-g{X)\dzdydX<tin + t2n, 

hn = /n2 Kn{X-z)f{z)\g{z)-g{X)\ dz dX, t2n = /ns Kn{X- z)Kn{y- X)f{z) x 
\g{y) — g{z) \ dXdzdy. It suflaces to show that \/nt2n = o(l) since arguments 
from (22) provide ^Jnt\n = o(l). Applying Lemma l(iv) and (20) sequen- 
tially, we bound /q Kn{X — z)Kn{y — A) dX by 



C 



Cn 



L„o(A - z)LUy -X)dX< _ ^^^^^^ ^^^^^^ . 

From this and arguments from (22), we have 

f{zMy)-g{z)\ 



V^t2n < Cr?!'' f - 
Jn2 (1 



dzdy = o(l). 



(1 + |(y — z) mod27r|n)2 

For the second part of (31), we use (27r/n) X]j=''-]v Cnff(Aj)/n,c(Aj) = 2Cr, 
and write 



2Vn 



27r 



N 



— Il5(Aj)/n(Aj) -C„ 



< 4:■^■^/n{t3n + t4:n), 
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t3n = n ^EjLl-^n(Aj)|c„5((Aj)-C/(Aj)|, t4n = n H|Cn.5'(0)|/„c(0) + |c„5((7r)| X 

^c(vr)). We have ^/EUn = Op(l) from E[4c(0)] < Cn", \cngiO)\ < Cn'^ and 
|cn5(7r)|E[/„c(7r)] < C by Assumptions A.1-A.2. For a C > independent of 
1 ^ J ^ A^ ('^ > 3), if we establish 



(32) \cng{Xj)-giX,)\<CX^' 



/3i/log(ra) l{i>n/4} 



+ 



n-2j 



l<j<N, 



then it will follow that \Jnt-},n < Cn~^/^ log(n)(maxi<j<7v /n(Aj)Aj ) I]j=i 
Op(l) from Lemma 8. 

Fix 1 < J < A^. To prove (32), we decompose the difference 



|cn9(Aj) -5(Aj)| 



^n{y){9{Xj -y) - g{Xj)]dy 



< 



d'jn{y)dy 



+ 



where df^{y) = Kn{y)[g{Xj +y)- g{Xj)], dj^{y) = Kn{y)[g{Xj -y)- g{Xj)\. 
We separately bound the last two absolute integrals using (20), \g{y) — 
g{z)\ < C lyj-^+'^ilz - y\ for Q<\y\< \z\ < vr and the fact that Xj > 7r/2 if 
and only if j > n/4. Using Lemma 1, we bound 



l/n 



d%{y)dy 



< cxj'+f^ 



l/n 



LMydy<cr^xf, 



djn{y)dy 

'l/n 

and if Xj < vr/2, 



< cx: 



ny 



l/n {l+nyy 



dy<CrHog{n)Xf, 



djn{y)dy 



<Cn-^Ll{7r/2) ldy<Cn-'^ 

J TT — A T 



A,. 



If Aj > ir/2, we use a substitution u = 2tt — Xj — y and the fact that 1 < 
n - 2j < A^ to find 

27r-2A, 



djniy)dy 



Kn{2TT -u- Xj)[g{u) - g{Xj)] du 



< CXf 



{u — Xj)n 



{1 + {2Tr - u - Xj)n) 



■dt/<C7n-Mog(n)Af 



dj'niy)dy 



TV — Xj 



<C I ' Kn{2TT -u-Xj)du<C{n-2jy^xf 



Hence, the bound in (32) applies to \ d'jj^dy\; the same can be shown 
for \ dj^dy\ by considering separate integrals over the intervals (0,1/n], 
(l/n,Aj72], (Aj/2,Aj] and (Aj,7r] if Aj > 7r/2, or {Xj,2Xj] and (2Aj,7r] if 
Xj < 7r/2. See Lemma 12 of [32]. □ 
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Lemma A. 2. Under Assumption A.l, suppose g,h are real-valued, even 
Riemann integrable functions on H such that \g{X)\, \h{X)\ < C\X\^ , < /3 < 
1, a- (3 < 1/2. Then asn^oo, Vn = Var[{27r/n)i2f=ig{Xj)h{Xj)I^{Xj)] = 

0(1). 

Proof. We expand Vn and then bound Vn < Vi„ + V2n with Vin and 
V2n defined as 

N 

-g5:(A,)^^cum(|d„,(A,)|Md.e(A,)r), 

E {X^X,f^\cum{\dnciXi)\^\dnciXJ)\% 

respectively. Let V be the set of all indecomposable partitions P of the 
labels in the two row table {agt}, s = 1,2, t = 1,2,3,4 ([5], Section 2.3). 
We write the elements of a partition P = (Pi, . . . ,Pfn), ^ ^rn<7, with 
each Pi C {a^J = \JZi Pi, PiCi Pj = if i ^ j. For l<i<j<N, we define 

^11 — ^12 — ~'^13 — ~^14: — ^21 — ^22 — ~^23 — ~'^24 — ^j- product 

theorem for cumulants, 

cum(|d„(Ai)|'', |(i„(Aj)|^) = cuuiijniP), 

P&V 

(33) 

m 

cvLm.ijn{P) = n cmn{dnc{a^t) ■ o-st G Pu)- 

u=l 

Because E{dnciX)) = 0, we need only consider those partitions V* = {P = 
(Pi, . . . ,Pm) G V ■ 1 < \Pi\ < • • • < \Pm\, 1 < m < 6} where each set in the 
partition has two or more elements ast, using to denote the size of a fi- 
nite set B. Defining C/i„(P) = n'^ EjLi(Aj)^^|cumjj„(P)| and C/2„(P) = 
n"'^Ei<i<j<Ar(^i^j)^^|cumij„(P)|, we can bound Vin < CJ^PeV* ^iniP), 
z = 1,2, so that it suffices to show 

(34) U^n{P)=o{l), Per*, i = 1,2. 

By Lemma 3, we have |cumjj„(P)| < Cn^Aj^" for P eV*,l < j < N so 

that Uin{P) < Cn"^'^^^0'2<^-2/5}-i(n-i ^^1^ Af -^") = o(l) since a- /3 < 1/2. 
Hence, (34) is established for Uin and Vin = o(l). 

We now show (34) for U2n{P) by bounding |cumjj„(P)|, over 1 < i < j < 
A'", for several cases of P = (Pi, . . . , Pm) S V*. These cases are m = 1; m = 2, 
IP2I = 6; m = 2, jPa] = 5; m = 2, jPa] = 4; m = 3, IP3I = IP2I =3; m = 3, 
IP3I = 4, IP2I = 2. The last (seventh) case m = 4, |Pi| = IP2I = IP3I = IP4I = 2 
has the following subcases: 
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(7.1) there exist h / /ca where EastePk^ ^'st = = Ea.tGPfc^ 

(7.2) there exists exactly one k where X^a^iGPfe "^si ~ 0; 

(7.3) for each k, T.ast&Pk ^ ^ ^^^'^^ ^^^^ ^i' ^2, I Ea.tsPfe, «il = 

(7.4) for each k, |Ea.,eP.4l {0, 2A,„ 2A,}. 

The first six cases follow from Lemma 3 and (20). For example, under 
cases 3 or 4, we apply Lemma 3(ii) and (20) twice to bound \cnm.ijn{P)\ < 
C{n3/2 + nlog^(n)}2(AiAj)-2° and then C/2n(P) < Cn-i(n-i EjLi Af"^")^ = 
o(l). See Lemma 13 of [32] for details. 

To treat case 7, we define some sets. For < p < 7r/2, let A = : 
l<i<j<N}, Ap = eA:Xj< p}, AP = {(i,i) G ^ : Aj > p}, = 

e A:i + j > n/2} and ^„/2 = {(i, j) € A : i + j < n/2}. We will also 
use the fact that for integers j > i> 1, 

12 12 

(35) r- < — if ?' = 1 or j(i — 1) > i^, otherwise, — < — . 

i{3-i) j i j 

For subcase 7.1, define Api = ^ Ap : i = 1 or j{i — 1) > i^} and Ap2 = 

Ap\Api. 

Subcase 7.1. Without loss of generality, say ki = 1, k2 = 2. We have 
I Ea.tGPa ^'st\ = I Ea.iGP4 4*1 ^ {Aj " Aj, Aj + Aj} and by Lemma 3(i), JlLi x 
|cum(d„,(4i) : a,t & Pk)\ < Cn^iXiXj)"' ■ Fix < p < 7r/2. If | Ea,,eP, 4l = 
\j — Aj, A; = 3,4, then by Lemma l(i). Lemma 2 and (20) (for the sum in- 
volving Xj > p) or Lemma l(i). Lemma 3(i) and (20) (for the sum involving 
Xj<p): 



A,- + 



+ ±_yAf-^"A2^-" 



uM < C{p) {o,a-« + {n~'Rn,? [ - E ] ] = 



AT 



with some fixed max{a, 1/2} <d<l. Using (35) on the sums over Api and 
Ap2, we have 

(n\^r(^ \2/3-2a^^ , C (AjAj)^^"^" C ^ (AjAj)^^"^" 



Xi<Xi<p ' Api ^-^ > Ap^ ^-^ ' 
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n 



Xi<Xi<p 



2f3-2a 



Then lim(ui„(p) + U2n{p)) < C^^ X^^-^"" d\f = C7p2+4/3-4a ^ ^ ^^^^ 
does not depend on < p < 7r/2. Hence, U2n{P) = o(l) since 2 + 4/3 — 4a > 0. 
If I ^aat&Pk '^^at\ = + -^j; ^ = 3,4, we essentially repeat the same steps as 
above to show U2n{P) < u^n{p) + u/^nip) , where u^nip) < Cu2nip), and, using 
Lemma l(i) to expand u^nip) = C{p)/n'^Y.Ap ^i~"[Lno{\ + \j) + Rnpf , 



usnip) < 



C{p) ( 

2 



n 



E 



A: 



l3-a 



{n-j-i)' 



+ E77^ + («np)^' 



ln/2 



re ^''^ 



Thus, U2n{P) =o(l) in this case. 

Subcase 7.2. Without loss of generality, we assume | X^a^iGPi '^si 



0(1). 



A,- + 



\j — Aj. Applying Lemma l(ii) with Lemma 3(i) gives 



nfc=3 \cnm.{dnc{agi) : a^t G Pk)\ < Cre2(AjAj) Using this, along with Lemma 2, 
Lemma 3(i) and (20) for a fixed < p < 7r/2, we may bound U2n{P) < 
'U5n(/o) +ii6n(/o), where 

C{p) 



U5r. 



rr 



A^ " [-^nO ( A.j + Aj ) + i2„p] 



AP 



n 



+ R 



11 p 



and uM = Cn-^EA^>^^^j?''-''K^'"[^J' + ^ni(A. + \,mj' + 
Lnii^i — Aj)]. By Lemma l(ii), we can show UQn{p) < Cu2n{p) while, for 
large re, u^n{p) is bounded by 



C{p) 



E 



re - J 

AT 



+ 



R 



np 



re 



+ E 



n/2 



+ 



np 



re 



+ -RnpE^i 

7=1 / 



< C{p) j n-l+'-^-{0,a-/3} ^^-2 ^ ^np 



TV 



re \ re 



0(1), 



using n — j — i> j — i above. Hence, U2n{P) = o(l). 



Subcases 7.3-7.4. For Subcase 7.3, there exists some k such that 
I^a«t6Pfc«sil = - or Aj + Xi- Note that \c\xm{dnc{a^t) '■ "-st G -PfcJI < 
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CnX^°', \cum{dnc{ast) '■ dgt S -^^2)1 ^ ^^'^J" Lemma 3(i). For Subcase 7.4, 
the possible partitions P are as follows \ J2ast&Pk'^^st\ ~ + ^'^^ each k 
(or Xj — Xi for each k) or there exist ki, k2 where | J2ast<^Pk ^^^1 ~ '^J ~'~ '^^^ 
\J2ast&Pk2 "^^^^ ~ ~ ^^Su™6^ts as in Subcases 7.1-7.2 can show that 
(34) holds; see Lemma 13 of [32]. □ 
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